Sync fork to TransformerEngine v2.17

This PR integrates upstream NVIDIA/TransformerEngine v2.17, preserving all multi-backend plugin architecture and device abstraction patches.

Summary

• Base: NVIDIA/TransformerEngine v2.17 (commit 2e559f06)
• Previous fork base: v2.14
• Changes: 233 files (82 modified, 151 added)
• Branch: sync/upstream-v2.17-squashed (single squashed commit: dcfe4d60)

Key Changes

1. Upstream Integration (v2.14 → v2.17)

Merged 3 months of upstream development, including:

• Expert Parallelism (EP) support with NCCL symmetric memory integration
• JAX backend improvements and multi-process EP tests
• Quantization enhancements: MXFP8 attention, NVFP4 support
• Fused operations: grouped MLP, fused router, multi-head cross-attention
• API evolution: 30+ new C++ bindings, deprecated ops cleanup

2. Conflict Resolution (25 files)

Preserved fork patches:

• torch → torch_nv framework extension loading (pytorch/init.py)
• te_device_type() device abstraction throughout codebase
• Plugin OpManager backend dispatcher architecture
• Multi-backend CI/CD infrastructure (9 backends: cuda/npu/hygon/kunlunxin/iluvatar/metax/musa/enflame/tsingmicro)

Accepted upstream refactors:

• Quantizer.make_empty() consolidated to base class (transformer_engine/pytorch/quantized_tensor.py)
• MoE permutation ops migrated to torch.library.custom_op (transformer_engine/pytorch/ops/permutation.py)
• Expert Parallel (EP) support + NCCL submodule integration

Device abstraction re-injection points:

• quantized_tensor.py:331 (base class make_empty)
• quantization.py:418 (set_skip_fp8_weight_update_tensor)
• Additional injections in attention, tensor, module layers

3. Plugin API Audit & P0 Fixes

Problem: Upstream PR#3001 deleted 2 C++ bindings that 6 vendor backends depended on:

• convert_host_pointers_to_tensor → replaced by copy_data_ptrs_to_device
• get_device_pointer_for_data_and_scales → replaced by transform_and_copy_data_ptrs_to_device

Solution: Removed dead registrations from plugin layer (13 files: plugin/core/ops.py + 6 vendor backends × 2 files each)

Affected backends: cuda, enflame, hygon, iluvatar, metax, musa

4. Enum Synchronization (3 files)

Root cause: Fork's plugin architecture defines IntEnum mirrors of C++ enums for reference backend, but v2.17 added strict equality checks between C++/Python enums + new enum members.

Fixes:

1. constants.py:83 - DType assert: == → <= (subset check)
   • Allows plugin superset (12 members including kInt16/kInt64 for reference backend) vs upstream's 8
2. plugin/core/ops.py - NVTE_QKV_Format: Added missing NVTE_BHSD=7, NVTE_QKV_Format_NOT_SET=8
3. plugin/core/ops.py - NVTE_QKV_Layout: Added missing NVTE_BHSD_BHSD_BHSD=25

5. Build Integration

Submodules:

• cudnn-frontend: 7b9b711c → e46d7082 (v2.17)
• cutlass: e64a9136 (fork, unchanged)
• googletest: a0f06a70 (fork, unchanged)
• nccl: a6b5de08 (NEW in v2.17 for EP support)

Build note: NCCL EP compilation requires NVTE_WITH_NCCL_EP=0 until submodule SHA updated to version with required APIs (ncclWinGetUserPtr, ncclGetPeerDevicePointer, etc.). This is an upstream deficiency, not introduced by this sync.

Verification

CUDA Backend (H100 hardware)

• ✅ import transformer_engine.pytorch successful (no AttributeError)
• ✅ LayerNorm forward pass: shape/dtype correct
• ✅ Linear forward pass: output correct
• ✅ OpManager correctly dispatches to vendor.cuda

Other Backends

• 🔲 NPU/Hygon/Kunlunxin/Iluvatar/Metax/MUSA/Enflame/Tsingmicro: Marked BLOCKED pending hardware availability

Code Quality

• ✅ All merge conflicts resolved (0 markers)
• ✅ Python AST validation passed for all .py files
• ✅ No P0 issues remaining in decisions.tsv

Artifacts

Detailed resolution logs and analysis artifacts available at:
/workspace/upgrade/artifacts/te-upstream-v2.17/

Key files:

• fork-delta.txt - Initial delta analysis (233 files)
• decisions.tsv - Decision log for all conflicts
• stage2-resolution-summary.txt - Conflict resolution details
• bindings.txt / plugin-base.txt / registered.txt - API surface audit
• api-matrix.tsv - Plugin API compatibility matrix

Testing Recommendations

1. CUDA regression: Run full test suite on NVIDIA hardware
2. Multi-backend smoke tests: Basic import + forward pass on each backend
3. EP functionality: Expert parallel tests (requires NCCL EP build fix first)
4. Quantization: MXFP8/NVFP4 attention tests

Notes

• This is a sync-only PR - no new features added by fork
• All fork-specific patches preserved and re-applied where upstream refactored
• Single squashed commit for clean main branch history
• Follow-up work needed: Update NCCL submodule SHA for full EP support

Verification command (CUDA):

python -c "import transformer_engine.pytorch as tep; print('✓ Import OK'); \
  ln = tep.LayerNorm(128).cuda(); x = torch.randn(2,8,128).cuda(); \
  y = ln(x); print(f'✓ Forward pass OK: {y.shape}')"